Color Separation for Background Subtraction

Background subtraction is a vital step in many computer vision systems. In background subtraction, one is given two (or more) frames of a video sequence taken with a still camera. Due to the stationarity of the camera, any color change in the scene is mainly due to the presence of moving objects. The goal of background subtraction is to separate the moving objects (also called the foreground) from the stationary background. Many background subtraction approaches have been proposed over the years. They are usually composed of two distinct stages, background modeling and foreground detection. Most of the standard background subtraction techniques focus on the background modeling. In the thesis, we focus on the improvement of foreground detection performance. We formulate the background subtraction as a pixel labeling problem, where the goal is to assign each image pixel either a foreground or background labels. We solve the pixel labeling problem using a principled energy minimization framework. We design an energy function composed of three terms: the data, smoothness, and color separation terms. The data term is based on motion information between image frames. The smoothness term encourages the foreground and background regions to have spatially coherent boundaries. These two terms have been used for background subtraction before. The main contribution of this thesis is the introduction of a new color separation term into the energy function for background subtraction. This term models the fact that the foreground and background regions tend to have different colors. Thus, introducing a color separation term encourages foreground and background regions not to share the same colors. Color separation term can help to correct the mistakes made due to the data term when the motion information is not entirely reliable. We model color separation term with L1 distance, using the technique developed by Tang et.al. Color clustering is used to efficiently model the color space. Our energy function can be globally and efficiently optimized with graph cuts, which is a very effective method for solving binary energy minimization problems arising in computer vision. To prove the effectiveness of including the color separation term into the energy function for background subtraction, we conduct experiments on standard datasets. Our model depends on color clustering and background modeling. There are many possible ways to perform color clustering and background modeling. We evaluate several different combinations of popular color clustering and background modeling approaches. We find that incorporating spatial and motion information as part of the color clustering process can further improve the results. The best performance of our approach is 97% compared to the approach without color separation that achieves 90%.